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We  are  providing  this  report  for  your  review  and  comments.  We  considered 
management  comments  on  a  draft  of  this  report  when  preparing  the  final  report. 
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Evaluation  of  the  Defense  Supply  Center  Columbus 
Qualified  Products  List  and 
Qualified  Manufacturers  List  Program 


Executive  Summary 

Introduction.  This  report  is  a  review  of  the  Defense  Supply  Center  Columbus 
Qualified  Products  List  and  Qualified  Manufacturers  List  Program.  The  Defense 
Supply  Center  Columbus  Qualified  Products  List/Qualified  Manufacturers  List  Program 
was  started  in  1962  when  qualifying  activity  responsibility  for  about  75  Qualified 
Products  Lists  was  transitioned  from  the  Military  Departments.  The  Defense  Supply 
Center  Columbus  currently  has  management  responsibility  for  over  300  Qualified 
Products  Lists  and  4  Qualified  Manufacturers  Lists.  The  Defense  Supply  Center 
Columbus  actively  procures  approximately  186,000  National  Stock  Numbers  from 
Qualified  Products  Lists  with  sales  of  over  $126  million  in  FY  2000.  The  Sourcing  and 
Qualifications  Unit  was  assigned  the  responsibility  of  managing  the  Qualified  Products 
List/Qualified  Manufacturers  List  Program.  Because  the  Defense  Supply  Center 
management  directed  the  Sourcing  and  Qualifications  Unit  to  shift  engineers  to  the 
Commodity  Management  Group  and  Base  Reutilization  and  Closure,  the  workforce  has 
been  reduced  from  approximately  90  staff-years  in  FY  1989  to  50  staff-years  in 
FY  2000.  In  the  same  time  period,  the  Qualified  Products  Lists  and  Qualified 
Manufacturers  Lists  managed  increased  from  164  to  over  300. 

Results.  Defense  Supply  Center  Columbus  did  not  have  an  effective  manufacturer  and 
product  qualification  list  program. 

•  Certification  and  retention  policy  for  its  qualified  manufacturers  was  not  fully 
enforced.  Specifically,  512  (42.8  percent)  of  the  required  1,196 
manufacturing  line  audits  scheduled  during  1999-2000  were  not  accomplished. 
Some  of  the  manufacturing  lines  have  gone  8  years  without  certification. 

•  The  Sourcing  and  Qualifications  Unit  did  not  receive  1,739  Product  Quality 
Deficiency  Reports  required  to  monitor  the  Qualified  Products  List  and 
Qualified  Manufacturers  List  Programs. 

•  The  Sourcing  and  Qualifications  Unit  could  not  trace  sources  and  authenticity 
of  their  products  because  they  did  not  receive  6,479  certificates  of 
conformance  and  traceability. 

As  a  result,  manufacturers  without  the  proper  certification  remained  in  the  Qualified 
Products  List  and  Qualified  Manufacturers  List  Program.  Further,  the  Services,  which 
rely  on  these  programs,  are  subjected  to  a  higher  risk  of  receiving  nonconforming 
parts.  For  details  of  the  evaluation  results,  see  the  Finding  section  of  this  report. 


Summary  of  Recommendations.  We  recommend  that  the  Commander,  Defense 
Supply  Center  Columbus,  fully  implement  the  procedures  specified  in  the  DoD  and 
Defense  Logistics  Agency  regulations  to  either  recertify  the  manufacturer  or  remove  it 
from  the  Qualified  Products  List/Qualified  Manufacturers  List  Program;  provide 
product  quality  deficiency  reports  and  certificates  of  conformance  and  traceability 
documentation  to  the  Defense  Supply  Center  Columbus  qualified  products  list 
managers;  and  perform  a  review  of  resources  required  to  accomplish  the  certification 
and  provide  adequate  resources  in  order  to  accomplish  the  mission. 

Management  Comments.  The  Director,  Defense  Logistics  Agency  partially  concurred 
with  the  finding  and  recommendations.  The  Director  did  not  concur  that  they  did  not 
have  an  effective  program  and  stated  that  the  program  effectiveness  should  have  been 
based  on  avoiding  procurement  delays.  They  partially  concurred  with  either  recertifying 
or  removing  manufacturers  from  the  lists  and  stated  that  the  policy  does  not  have  such  a 
provision.  The  Defense  Logistics  Agency  partially  concurred  with  providing  product 
quality  deficiency  reports  and  certificates  of  conformance  and  traceability  to  Qualified 
Products  List  and  Qualified  Manufacturers  List  managers  and  proposed  implementing 
actions,  and  concurred  with  performing  a  review  of  resources  to  accomplish  the 
certifications.  A  discussion  of  management  comments  is  in  the  Finding  section  of  the 
report  and  the  complete  text  is  in  the  Management  Comments  section. 


Evaluation  Response.  Shorter  lead  times  are  included  in  the  Qualified  Products  List 
and  Qualified  Manufacturers  List  Program  objectives  as  is  improved  readiness  through 
availability  of  reliable  products.  Our  evaluation  focused  on  the  latter,  compliance  with 
Qualified  Products  List  and  Qualified  Manufacturers  List  requirements  as  part  of  the 
Quality  Assurance  Programs.  The  Defense  Logistics  Agency  does  have  the  authority  to 
remove  manufacturers  from  the  Qualified  Products  List  and  Qualified  Manufacturers 
List  when  it  is  necessary  to  protect  the  Government  and  the  users’  interest.  In  the 
response  to  the  final  report,  we  request  that  the  Defense  Logistics  Agency  reconsider 
their  position  and  provide  comments  to  the  recommendation  to  remove  manufacturers 
until  their  qualification  audits  are  current.  The  Defense  Logistics  Agency  proposals  for 
providing  product  quality  deficiency  reports  and  certificates  of  conformance  and 
traceability  to  Qualified  Products  List  and  Qualified  Manufacturers  List  managers, 
along  with  reported  corrective  actions  underway,  meet  the  intent  of  our 
recommendations.  The  Defense  Logistic  Agency  proposed  review  of  resources 
required  to  accomplish  the  certifications  meets  the  intent  of  our  recommendation. 

In  response  to  the  final  report,  we  request  that  the  Defense  Logistics  Agency  provide 
completion  dates  for  the  review  of  engineering  resources  required  to  accomplish  the 
certifications.  We  request  that  the  Defense  Logistics  Agency  provide  the  additional 
comments  on  Recommendations  1.  and  4.  by  July  15,  2002. 
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Background 


Section  2452,  title  10,  United  States  Code  (10  U.S.C.  2452),  “Duties  of  the  Secretary 
of  Defense,”  requires  the  Secretary  to  develop  and  maintain  the  supply  catalog  and  the 
standardization  program  to  establish,  publish,  review,  and  revise,  within  the 
Department  of  Defense,  military  specifications,  standards,  and  lists  of  qualified 
products  as  described  in  10  U.S.C.  2451.  The  Federal  Acquisition  Regulation  (FAR) 
subpart  9.2,  “Qualifications  Requirements,”  implemented  10  U.S.C.  2319, 
“Encouragement  of  New  Competitors,”  and  41  U.S.C.  253c,  “Encouragement  of  New 
Competition,  ”  and  prescribed  policies  and  procedures  regarding  qualification 
requirements  and  acquisitions  that  were  subject  to  such  requirements.  Sections  9.201 
and  9.203  of  the  FAR  defined  and  described  the  Qualified  Products  List  (QPL)  and 
Qualified  Manufacturers  List  (QML)  Program. 

The  QPL  and  QML  Program  aims  to  increase  product  quality,  reliability,  and  buying 
productivity,  and  enhance  logistics  management  operations  by  establishing  a  list  of 
products  that  have  met  the  qualification  requirements  stated  in  the  applicable 
specification.  The  requirements  will  also  include  appropriate  product  identification  and 
test  or  qualification  reference  with  the  name  and  plant  address  of  the  manufacturer  and 
distributor  as  applicable  for  select  commodities.  Instead  of  source  inspections  and 
product  verifications,  quality  levels  are  maintained  through  initial  and  periodic 
certification  of  the  manufacturer  by  the  Government,  use  of  deficiency  reporting 
information  provided  by  the  user,  and  product  quality  control  procedures  maintained  by 
the  manufacturer.  The  intended  results  are  shorter  lead  times  for  acquisition  and 
procurement,  reduced  test  costs,  and  improved  readiness  through  continuous  availability 
of  reliable  products  from  viable  suppliers.  Policies  and  procedures  for  the  Qualification 
Program  (QPL  and  QML)  are  contained  in  Appendix  B  of  “Defense  Standardization 
Program  (DSP)  Policies  and  Procedures,”  DoD  Manual  4120. 24-M,  March  2000. 
Definitions  of  technical  terms  for  the  QPL  and  QML  Program  are  in  Appendix  B,  and 
some  of  its  salient  features  are  described  as  follows. 

Qualifying  Potential  Products,  Processes,  and  Materials.  Qualification  is  the  process 
by  which  products,  processes,  or  materials  of  manufacturers  or  distributors  are 
independently  examined  and  tested  to  determine  whether  they  conform  to  specification 
requirements  before  they  are  acquired.  Products  and  manufacturers  that  successfully 
pass  the  qualification  process  are  then  identified  on  a  list  of  qualified  products  or 
qualified  manufacturers.  Criteria  to  retain  qualification  are  applied  periodically  to 
ensure  continued  integrity  of  the  qualification  status. 

Maintaining  the  Product  Quality.  Once  a  product  or  manufacturer  is  added  to  a  QPL 
and  QML,  the  manufacturer,  the  user,  and  the  Government  are  responsible  for 
maintaining  product  quality.  The  manufacturer  maintains  adequate  process  and  quality 
control  procedures  and  conducts  specified  testing  to  ensure  that  items  continually 
comply  with  all  specification  requirements.  In  addition,  the  manufacturer  maintains 
trend  analysis  and  reports  of  deficiencies  disclosed  during  testing,  and  ensures  that 
delivered  items  conform  to  all  requirements.  The  user  ensures  that  the  qualified 
products  delivered  comply  with  the  specification  requirements  by  reporting  any 
nonconformances  to  the  qualifying  activity  (QA)  and  by  submitting  periodic  summaries 
of  quality  control  monitoring  results  that  reveal  adverse  quality  and  reliability  trends  to 
the  QA.  The  QA  is  responsible  for  the  overall  management  of  the  QPL  and  QML 
Program  and  must  verify  the  manufacturer’s  compliance  to  the  specified  requirements. 
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The  QA  is  also  required  to  take  appropriate  action  on  any  reported  nonconformance  and 
quality  or  reliability  problems  associated  with  QPL  and  QML  products. 

Qualified  Products  List.  A  QPL  will  normally  be  appropriate  for  products  of  supply, 
which  have  a  stable  design  and  composition  and  will  be  continually  available  for 
extended  periods  of  time,  thereby  making  it  practicable  to  qualify  individual  products 
without  incurring  prohibitive  testing  costs.  A  product  that  meets  the  established 
qualification  requirement  will  be  listed  on  the  QPL.  The  QPL  contains  in  most  cases 
less  complex  products  such  as  cables,  resistors,  or  hoses. 

Qualified  Manufacturers  List.  A  QML  focuses  on  qualifying  a  manufacturer’s 
materials  and  processes  rather  than  products.  A  QML  will  normally  be  appropriate  for 
items  of  supply  that  are  experiencing  very  rapid  technological  advances  or  have  a 
myriad  of  variations  or  custom  designs  that  make  individual  product  qualification 
impractical  or  excessively  expensive.  A  QML  applies  to  processes  or  materials  that 
generally  meet  the  following  criteria: 

•  they  do  not  have  recognized  industry  part  numbers; 

•  they  are  procured  to  a  specification  that  covers  a  wide  range  of  technologies, 
like  hybrid  microcircuits;  and 

•  they  are  a  family  of  products  with  similar  characteristics,  like  printed  wiring 
boards . 

Representative  worst  case  test  vehicles  or  representative  samples  that  contain  all 
potential  combinations  of  materials  and  processes  used  during  production  are  carefully 
examined  in  order  to  determine  acceptability  limits.  As  evidence  that  those  processes 
and  materials  meet  the  established  qualification  requirements,  the  envelope  of 
acceptance  processes  and  materials  will  be  listed  on  a  QML.  For  example,  for 
microcircuits  that  require  ceramic  capability,  when  an  order  is  placed,  the  manufacturer 
will  ensure  that  all  certification,  qualification,  and  documentation  items  are  complete  as 
required  by  the  Defense  Supply  Center  (DSC)  Columbus  approved  quality  management 
plan.  The  manufacturer  will  also  submit  to  the  qualifying  activity  a  certificate  of 
compliance  for  the  applicable  standard  microcircuit  drawing.  Upon  completion  of  these 
requirements,  the  product  listing  information  will  be  transferred  to  the  QPL  as  a 
qualified  product. 

QPL  Review  and  Certification.  The  qualifying  activity  must  review  specifications 
that  require  qualification  every  2  years  to  determine  the  need  to  continue  the 
qualification  requirement.  In  addition,  the  qualifying  activity  is  responsible  for 
reviewing  and  certifying  on  a  periodic  basis  each  listed  manufacturer.  Furthermore,  the 
qualifying  activity  shows  whether  more  definitive  product  requirements,  advances  in 
manufacturing  techniques  and  quality  control  methods,  or  improvements  in  testing 
apparatus  and  techniques  have  eliminated  the  need  for  qualification.  The  certification 
process  also  determines  whether  the  listed  products  are  manufactured  at  the  same  plants 
shown  on  the  listing  and  whether  they  are  being  manufactured  under  the  same 
conditions  as  when  first  qualified.  The  process  also  determines  whether  the  plants  have 
remained  under  the  same  management  and  manufacturing  controls.  To  obtain 
qualification  approval  of  products,  one  of  the  following  actions  is  required: 
a.  certification  by  the  manufacturer  (submittal  of  a  DD  1718  form  “Certification  of 
Qualified  Products”);  b.  periodic  submission  of  new  test  data,  as  may  be  required  in  the 
specification;  or  c.  complete  re  qualification  testing,  as  required  in  the  specification  or 
by  the  QA. 
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QML  Review  and  Certification.  The  qualifying  activity  is  responsible  for  reviewing 
and  recertifying  on  a  periodic  basis  the  continued  need  for  each  QML.  In  addition,  the 
qualifying  activity  must  review  specifications  that  require  qualification  every  2  years  to 
determine  the  need  to  continue  the  qualification  requirement.  QML  includes  reviewing 
the  manufacturer’s  annual  retention  of  qualification  reports  according  to  the 
requirements  of  the  applicable  specification,  performing  periodic  facility  audits,  and 
reviewing  design  and  construction  or  other  changes  made  by  the  manufacturer  to  the 
qualified  products  as  delineated  in  the  applicable  military  specifications.  This  review 
process  helps  assure  continued  manufacturer  compliance  and  up-to-date  qualification 
requirements  (including  associated  quality  conformance  and  reporting  requirements). 
The  process  also  determines  whether  the  plants  have  remained  under  the  same 
management  and  manufacturing  controls. 

QPL  and  QML  Program  at  Defense  Supply  Center  Columbus.  The  QPL  Program 
at  DSC  Columbus  began  in  1962  when  qualifying  activity  responsibility  for  about  75 
QPLs  was  transitioned  from  the  Military  Departments.  From  1962  until  1989,  the 
number  of  QPLs  assigned  to  DSC  Columbus  increased  to  about  164.  After  acquisition 
reform,  the  number  of  QPLs  and  QMLs  assigned  to  DSC  Columbus  nearly  doubled, 
due  to  the  Secretary  of  Defense’s  decision  to  have  buying  commands  like  DSC 
Columbus  assume  more  control  over  the  products  they  actually  procure  for  the 
Services  and  Defense  agencies.  DSC  Columbus  currently  has  management 
responsibility  for  approximately  300  QPLs  and  4  QMLs.  Over  one  million  individual 
part  numbers  are  qualified  and  listed  on  the  applicable  DSC  Columbus  managed  QPLs 
and  QMLs.  DSC  Columbus  actively  procures  about  186,000  national  stock  numbers 
(NSNs)  from  about  900  qualified  manufacturers  and  distributors  with  sales  of  over 
$126  million  in  FY  2000.  These  qualified  NSNs  include  monolithic  microcircuits, 
hybrid  microcircuits,  aircraft  hydraulic  hoses,  vacuum  and  de-icing  system  components, 
resistors,  capacitors,  filters,  circuit  breakers,  switches,  connectors,  relays,  fiber  optic 
conductors  and  cables,  semiconductors  devices,  antennas,  waveguides,  and  electrical 
boards. 

DSC  Columbus  primary  management  of  the  QPLs  and  QMLs  is  the  responsibility  of 
the  Sourcing  and  Qualifications  Unit,  which  is  comprised  of  four  teams  that  contribute 
to  the  maintenance  of  its  QPLs  and  QMLs.  These  are  the  Custom  Devices  Team, 
Electronic  Devices  Team,  Hybrid  Devices  Team,  and  Passive  Devices  Team.  These 
teams  follow  DoD  Manual  4120.24-M  to  manage  their  QPL  and  QML  Programs.  The 
teams  are  responsible  for  administering  the  qualification  program,  maintaining  and 
publishing  QPL  and  QML  listings,  and  ensuring  the  manufacturers’  compliance  with 
qualification  requirements.  In  addition,  the  applicable  specifications  include  the 
detailed  qualification  and  retention  requirements  to  assure  continuous  product 
performance,  quality,  and  reliability.  All  specifications  and  corresponding  QPLs  and 
QMLs  are  maintained  in  the  DoD  Acquisition  Streamlining  and  Standardization 
Information  System  (ASSIST). 

The  Plans  and  Policy  Division  serves  as  the  focal  point  for  receiving  complaints  in  the 
form  of  Product  Quality  Deficiency  Reports  (PQDRs),  providing  entry  to  the  Customer 
Depot  Complaint  System,  and  distributing  the  complaints  to  an  appropriate  quality 
assurance  specialist  (QAS)  for  investigation,  resolution,  and  response.  Each  QAS 
provides  quality  assurance  support  to  the  product-buying  units  that  are  assigned  the 
management  and  procurement  responsibilities  for  a  group  of  NSNs,  including  QPLs. 
Upon  receiving  a  complaint,  the  QAS  assesses  the  need  for  investigation,  initiates  the 
necessary  corrective  actions  for  the  reported  defective  items,  and  coordinates  with  other 
elements  and  components  within  DSC  Columbus  as  applicable. 
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DSC  Columbus  has  in  the  past  worked  conscientiously  to  promote  the  use  of  the  QPL 
and  QML  Program  by  the  Services,  and  Federal  and  civilian  agencies.  The  QPL  and 
QML  Program  within  DSC  Columbus  Operations  Support  Group  is  consistently  applied 
across  the  various  industry  sectors.  The  Sourcing  and  Qualifications  Unit  consists  of 
highly  trained  engineers  and  experienced  technicians. 

Objective 


Our  objective  was  to  evaluate  the  Defense  Supply  Centers’  Quality  Assurance 
Programs.  Specifically,  we  evaluated  the  effectiveness  of  the  Defense  Supply  Center 
Columbus  Qualified  Products  List  and  Qualified  Manufacturers  List  Program.  The 
evaluation  also  reviewed  the  management  control  program  as  it  related  to  the  evaluation 
objective.  See  Appendix  A  for  the  scope  and  methodology,  management  control 
program  review,  and  a  summary  of  prior  coverage. 
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Effectiveness  of  Management  of  the  Qualified 
Products  List  and  Qualified  Manufacturers 
List  Program 

DSC  Columbus  did  not  have  an  effective  Qualified  Products  List  and  Qualified 
Manufacturers  List  Program.  It  did  not  fully  enforce  the  certification  and 
retention  policy  for  its  qualified  manufacturers.  A  total  of  512  (42.8  percent)  of 
the  required  1,196  qualification  product  manufacturer  facility  and  lines  audits, 
scheduled  during  1999-2000,  were  not  accomplished.  Its  Sourcing  and 
Qualifications  Unit  did  not  receive  1,739  Product  Quality  Deficiency  Reports 
required  to  monitor  the  QPL  and  QML  Program.  DSC  Columbus  could  not  trace 
sources  and  authenticity  of  their  products  because  DSC  Columbus  did  not  receive 
6,479  Certificates  of  Conformance  and  Traceability  (COCTs)  from  dealers  and 
distributors.  As  a  result,  manufacturers  without  the  proper  certification  remained 
in  the  QPL  and  QML  Program.  Further,  the  Services,  which  rely  on  these 
programs,  are  subjected  to  a  higher  risk  of  receiving  nonconforming  parts. 

Retention  of  Qualification  Policy 


DoD  has  established  policy  and  guidance  in  DoD  Manual  4120. 24-M  for  a 
manufacturer’s  certification  and  retention  of  qualification.  The  process  involves  the 
qualifying  activity’s  certification  of  each  manufacturer’s  qualification  status  every 
2  years.  To  retain  qualification  approval  of  products,  one  of  the  following  actions  is 
required: 

•  certification  by  the  manufacturer, 

•  periodic  submission  of  new  test  data  as  may  be  required  in  the  specification, 
or 

•  complete  requalification  testing  as  may  be  required  in  the  specification  or  by 
the  qualifying  activity. 

The  Qualifying  Activity  shall  determine,  based  on  the  extent  of  specification  or 
product  changes  and  odier  available  data,  whedier  products  or  manufacturers 
need  to  be  removed  from  the  QPL  or  QML  until  retested.  If  the  Qualifying 
Activity  determines  that  die  product  or  manufacturer  should  remain  on  die 
QPL  or  QML,  the  Qualifying  Activity  shall  establish  a  maximum  time  limit 
for  submission  of  the  samples  or  test  data  before  removal.  The  Qualifying 
Activity  shall  require  the  reexamination  of  a  qualified  product  when  required 
by  retention  of  qualification  requirements  hi  the  specification. 


Results  of  Retention  of  Qualification  Process  at  DSC  Columbus 

The  Sourcing  and  Qualifications  Unit  at  DSC  Columbus  stated  that  it  conducts  audits  on 
a  scheduled  basis  every  1-5  years  depending  on  the  specification  requirements.  Some 
specifications  call  for  retention  every  year  for  such  items  procured  by  DSC  Columbus 
for  the  National  Aeronautical  and  Space  Administration,  which  demands  very  stringent 
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safety  requirements.  The  process  involves  facility  audits  by  DSC  Columbus  engineers 
in  accordance  with  applicable  specifications.  At  the  conclusion  of  an  audit,  the 
manufacturer  is  required  to  take  corrective  action  on  any  deficiencies  found.  After  the 
Sourcing  and  Qualifications  Unit  accepts  the  corrective  actions,  the  manufacturer  is 
considered  “certified,”  and  retains  its  qualification  status  on  a  QPL  or  QML.  The  Unit 
also  receives  retention  reports,  product  failure  notices,  and  product  test  data,  which 
form  a  basis  for  continuing  a  manufacturer’s  QPL  or  QML  product  qualification. 

Results  of  Retention  of  Qualification  Assessment 

We  reviewed  the  certification  status  of  the  qualified  manufacturers  managed  by  four 
teams  within  the  Sourcing  and  Qualifications  Unit:  Customs  Devices  Teams  (VQC), 
Electronics  Devices  Team  (VQE),  Hybrid  Devices  Team  (VQH)  and  Passive  Devices 
Team  (VQP).  The  teams  were  responsible  for  requalifying  1,196  manufacturing  lines 
during  the  1999-2000  2-year  cycle.  VQC  was  responsible  for  requalifying  450  of  the 
manufacturing  lines  during  its  2-year  certification  cycle,  but  conducted  audits  for  only 
210  manufacturing  lines  during  that  period  of  time.  As  seen  in  Figure  1,  the  other 
branches  had  similar  results.  VQH,  VQE,  and  VQP  did  not  meet  their  requirements 
for  the  requalification  of  178,  216,  and  352  manufacturing  lines  respectively. 


VQC  VQH  VQP  VQE 


□  Manufacturers  line 
audits  required  every  2 
years 

■  Audit  Capability  every 
2  years 

□  Manufacturers  lines  not 
covered  every  2  years 


Figure  1.  Sourcing  and  Qualifications  Unit  QPL/QML 
Audit  Requirements  Versus  Capability 


The  1,196  manufacturing  lines  requiring  certification  in  the  2-year  cycle  were 
composed  of  684  for  the  QML  Program  and  512  for  the  QPL  Program.  Our  review 
determined  that  286  or  41.81  percent  of  the  QML  manufacturing  lines  were  not 
audited.  Similarly,  226  or  44.14  percent  of  the  QPL  manufacturing  lines  were  not 
audited.  In  total  512  or  42.80  percent  of  manufacturing  lines  that  needed  certification 
were  not  audited.  We  also  found  that  some  of  the  manufacturing  lines  have  gone 
8  years  without  certification. 
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DSC  Columbus  Sourcing  and  Qualifications  Unit  Personnel  and 
Workload 


A  review  of  the  QA  staffing  demonstrated  that  a  significant  reduction  in  personnel  had 
occurred  since  1989.  Moreover,  during  the  same  period,  their  QPL  and  QML 
workload  had  increased  significantly.  As  seen  in  Figure  2,  the  number  of  positions 
associated  with  the  program  decreased  from  90  in  1989  to  50  at  the  end  of 
FY  2000.  Conversely,  the  QPL  and  QML  workload  managed  by  the  QA  increased 
dramatically  over  the  same  period,  from  164  QPL/QML  in  1989  to  around  300  at  the 
end  of  FY  2000.  Currently,  the  assigned  resources  have  to  administer  the  QPL  and 
QML  Program,  accomplish  a  wide  range  of  engineering  certification  tasks  to  maintain 
the  QPL  and  QML  Program,  and  respond  to  new  qualification  applications.  The 
Sourcing  and  Qualifications  Unit  staff  stated  that  the  lack  of  staffing  is  the  cause  for 
outdated  certifications. 


□  QPL/QML 
■PERSONNEL 


YEARS 


Figure  2.  QPL/QML  Workload  Versus  Personnel 


Quality  Assurance  Policy  Regarding  PQDRs 


DoD  Manual  4120. 24-M  mandates  that  the  buying  activity  provides  feedback  based  on 
field  information  to  the  QA.  Furthermore,  DSC  Columbus  Integrated  Policy 
Memorandum  IPM-97-0048  establishes  a  QPL  and  QML  Quality  Assurance  Policy 
which  details  the  program’s  use  of  PQDRs: 

Quality  associates  will  immediately  notify  DSCC-VA  [Document  Standardization  Unit] 
and  VQ  [Sourcing  and  Qualifications  Unit]  of  all  validated  reports  of  component  failures 
or  other  quality  problems  (Product  Quality  Deficiency  Reports  and  validated  defective  test 
results)  which  may  impact  DSCC-V  [Operations  Support  Group]  administered  QPL/QML 
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products.  DSCC-VA  and  VQ  shall  alert  the  QAS  on  actions  taken  as  a  result  of  die  initial 
report  of  component  failure  or  report  of  other  quality  problems. 


PQDR  Role  in  the  Quality  Assurance  Process 


The  PQDRs  are  the  primary  source  for  the  QPL  and  QML  system  feedback  for  DLA 
and  the  Services  on  quality.  It  is  an  important  quality  control  measure  inherent  to  the 
success  of  the  QPL  and  QML  Program.  The  quality  assurance  specialists  in  the  Plans 
and  Policy  Division  are  required  to  review  each  PQDR  to  determine  whether  additional 
acceptance  testing  would  be  required  on  a  product  and  take  the  appropriate  corrective 
action  with  a  manufacturer.  Information  regarding  quality-related  problems  with 
qualified  products  is  also  important  for  maintaining  the  QPLs  and  QMLs.  However,  at 
DSC  Columbus,  problems  existed  with  the  ability  of  the  Sourcing  and  Qualifications 
Unit  to  acquire  such  feedback  because  the  policy  for  quality  assurance  specialists  to 
provide  PQDRs  to  the  Sourcing  and  Qualifications  Unit  was  not  enforced.  For 
example,  in  the  year  2000,  1,739  QPL  and  QML  related  PQDRs  were  submitted  to 
DSC  Columbus  but  were  not  provided  to  the  Sourcing  and  Qualifications  Unit.  As  a 
result,  QPL  and  QML  management  could  not  adequately  monitor  its  product 
deficiencies  and  could  not  take  corrective  actions. 

Qualification  Program  Policy  Regarding  Certificate  of 
Conformance  and  Traceability 


The  requirement  for  COCT  is  documented  in  several  of  the  critical  high  reliability 
military  specifications  in  order  to  ensure  that  an  authentic  QPL  or  QML  part  is  received 
when  ordered.  The  COCT  requirement  provides  traceability  from  the  original  QPL  or 
QML  qualified  manufacturer  to  the  buying  command,  agency,  or  user  of  the  part.  The 
COCT  documentation  attempts  to  reduce  the  risk  of  unauthorized  substitutes  and  fraud 
from  dealers  and  distributors.  Furthermore,  it  provides  the  QA  with  tools  to  determine 
if  qualified  parts  were  produced  by  a  qualified  manufacturer,  thereby  reducing  the 
incidence  of  part  substitutions  and  counterfeit  material. 

DSC  Columbus  local  policy  E-25  covers  the  relationship  between  the  qualification 
program  and  the  use  of  COCT.  The  traceability  clause  in  the  E-25  policy  is  used  as  a 
tool  for  quality  control  in  the  QPL  and  QML  Program.  This  policy  is  attached  to  each 
DSC  Columbus  contract  and  mandates  that  dealers  submit  three  copies  of  certification 
to  the  Government  Quality  Assurance  Representative  (QAR)  with  the  products  offered 
for  acceptance.  After  the  QAR  has  accepted  the  products  offered,  acceptance  and 
approval  of  the  traceability  certifications  is  indicated  by  signing  two  copies  of  the 
certification,  also  ensuring  that  the  contract  number  is  included  on  both  copies.  The 
contractor  is  mandated  to  provide  these  two  copies  to  DSC  Columbus.  The  third  copy 
must  be  retained  by  the  QAR. 

Review  of  Certificate  of  Conformance  and  Traceability  Status  at 
DSC  Columbus 


At  DSC  Columbus,  two  types  of  procurement  inspection  processes  are  used.  These  are 
designated  as  destination  and  source  inspections.  A  Defense  Contractor  Management 
Agency  QAR  inspector  provides  inspection  at  the  source  to  check  the  product  for 
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authenticity.  Destination  inspection  does  not  involve  the  Government  inspector  but 
relies  on  the  user  to  obtain  a  copy  of  the  COCT.  Currently,  approximately  60  percent 
of  the  QPL  items  are  designated  for  destination  inspection.  DSC  Columbus  Sourcing 
and  Qualifications  Unit  did  not  receive  copies  of  COCTs  from  dealers  and  distributors. 
For  example,  in  the  year  2000,  6,479  COCTs  were  required  at  DSC  Columbus  but  the 
Sourcing  and  Qualifications  Unit  received  none. 

Additionally,  a  recent  audit  conducted  by  the  DSC  Columbus  internal  review  office 
investigated  traceability  problems  in  the  QPL  process.  The  audit  team  selected  625 
contracts  from  dealers  and  distributors  and  conducted  a  random  selection  of  100 
contracts.  The  audit  found  that  only  6  out  of  100  contracts  contained  COCT 
documentation.  The  audit  showed  that  dealers  were  unable  to  provide  the  required 
COCT  from  the  original  manufacturers. 

Therefore,  the  Sourcing  and  Qualifications  Unit  was  unable  to  determine  the 
authenticity  of  the  product  and  recommended  not  accepting  products  from  dealers  or 
distributors.  As  a  result,  the  Sourcing  and  Qualifications  Unit  cannot  take  appropriate 
action  against  a  noncompliant  dealer  or  distributor. 

Summary 


DSC  Columbus  did  not  have  an  effective  QPL  and  QML  Program  because  it  did  not 
fully  enforce  the  certification  and  retention  policy  for  its  qualified  manufacturers. 

The  Sourcing  and  Qualifications  Unit  did  not  conduct  the  necessary  number  of  line 
audits  associated  with  its  qualified  manufacturers.  As  a  result,  42.8  percent  of  the 
required  1,196  qualification  product  manufacturer  facility  and  lines  audits,  scheduled 
during  1999-2000,  were  not  accomplished.  Further,  the  Sourcing  and  Qualifications 
Unit  at  DSC  Columbus  was  not  receiving  PQDRs  and  COCTs  because  the  policy  was 
not  enforced.  Without  reviewing  PQDRs  and  COCTs,  the  Sourcing  and  Qualifications 
Unit  could  not  effectively  monitor  product  quality,  authenticity,  or  source  and  could  not 
take  corrective  actions  against  nonconforming  QPL  and  QML  contractors.  As  a  result, 
some  quality  control  aspects  of  the  qualification  program  have  become  ineffective. 
Products  from  qualified  manufacturers  that  have  not  been  recertified  during  the  required 
requalification  cycle  may  be  at  a  higher  risk  of  not  conforming  to  the  required 
specifications.  Customers  receiving  items  procured  from  the  QPL  and  QML  Program 
may  have  a  false  sense  of  reliability  or  quality  and  may  be  subjected  to  an  increased 
risk  of  receiving  nonconforming  parts. 

Management  Comments  on  the  Finding  and  Evaluation  Response 


The  Defense  Logistics  Agency  partially  concurred  with  the  finding,  but  did  not  concur 
with  “DSC  Columbus  did  not  have  an  effective  manufacturing  and  product  qualification 
program.  ”  They  stated  that  they  schedule  and  conduct  facility  audits  within  resource 
allocations  and  established  priorities.  They  further  stated  that  the  effectiveness  of  the 
QPL  and  QML  Program  should  be  based  on  how  well  it  facilitates  the  “procurement 
process  by  qualifying  parts  and/or  manufacturers  in  advance  of  actual  procurement  or 
contracts,  thereby  avoiding  delays  .  .  .”. 

DLA  suggested  that  we  change  “recertification”  to  “certification”  since  “certification  is 
the  terminology  used  in  DoD  4120. 24-M.”  They  also  suggested  that  we  change 
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“preparing  activity”  to  “qualifying  activity”  since  the  qualifying  activity  is  responsible 
for  reviewing  and  certifying  each  QPL  and  QML  manufacturer  on  a  periodic  basis. 


Evaluation  Response.  Shorter  lead  times  are  included  in  the  QPL  and  QML  Program 
objectives.  Likewise,  improved  readiness  through  continuous  availability  of  reliable 
products  from  viable  suppliers  is  also  an  objective.  Our  evaluation  focused  on  the 
compliance  with  QPL  and  QML  requirements  as  part  of  the  Defense  Supply  Center’s 
Quality  Assurance  Programs.  Our  finding  and  its  causes  are  documented  in  the  body  of 
the  report. 

We  incorporated  the  suggested  changes  relative  to  using  “certification”  instead  of 
“recertification”  and  “qualifying  activity”  instead  of  “preparing  activity.” 

Recommendations,  Management  Comments,  and  Evaluation 
Response 


We  recommend  that  the  Commander,  Defense  Supply  Center  Columbus,  Defense 
Logistics  Agency: 

1.  Fully  implement  the  procedures  specified  in  DoD  Manual  4120.24-M,  “Defense 
Standardization  Program  (DSP)  Policies  and  Procedures,”  March  2000,  for 
Qualified  Products  List  and  Qualified  Manufacturers  List  Programs,  to  either 
certify  the  manufacturer  within  the  required  cycle  or  remove  the  manufacturer 
from  the  Qualified  Products  List  and  Qualified  Manufacturers  List  Program. 

Management  Comments.  DLA  partially  concurred  with  the  recommendation,  stating 
that  DSC  Columbus  “plans  their  product  qualification  audits  within  resource  allocations 
based  upon  the  prioritization  criteria,”  that  they  “have  resource  constraints  that  must  be 
managed,”  and  that  “engineering  resources  will  be  continually  reviewed.”  However, 
they  also  stated  that  DoD  policy  does  not  include  a  provision  for  them  to  “remove 
manufacturers  from  a  QML/QPL  solely  because  the  QA  did  not  perform  a  facility 
(plant)  audit,  nor  ‘recertify’  the  manufacturer  within  the  required  cycle.” 

Evaluation  Response.  Authority  to  remove  manufacturers  from  the  QML  and  QPL 
Program  is  contained  in  DoD  Manual  4120.24-M,  which  states  in  part: 

The  Qualifying  Activity  may  remove  a  product,  a  manufacturer,  or  a  process; 
decertify  a  manufacturer;  or  stop  shipment,  when  such  action  is  necessary  to 
protect  both  the  Government’s  interest  and  the  interest  of  the  users  of  the  QPL 
or  the  QML. 

The  manual  illustrates  circumstances  under  which  removal  “might”  be  warranted. 

While  the  illustrated  examples  of  circumstances  do  not  specifically  include  the 
Government  lack  of  compliance  with  the  certification  cycle,  the  intent  to  keep  only 
currently  qualified  products  or  manufacturers  on  the  list  is  illustrated  by  the  example  of 
removal  when  “the  manufacturer  has  not  complied  with  the  retention  of  qualification 
requirements.” 

DLA  acknowledged  that  “qualification  audits  are  important  tools  to  mitigate  quality 
problems  and  assure  continued  product  performance,  quality  and  reliability.” 

However,  with  42.80  percent  of  manufacturing  lines  that  needed  certification  not  being 
audited  and  with  some  manufacturing  lines  having  gone  8  years  without  certification, 
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removing  the  manufacturer  from  the  QPL  and  QML  Program  until  the  certification  is 
accomplished  meets  the  policy  criteria  of  being  necessary  to  protect  both  the 
Government’s  interest  and  the  interest  of  the  users  of  the  QPL  or  the  QML. 

In  response  to  the  final  report,  we  request  that  DLA  reconsider  their  position  of  not 
removing  manufacturers  from  the  Qualified  Products  List  and  Qualified  Manufacturers 
List  until  their  qualification  audits  are  current. 

2.  Provide  Product  Quality  Deficiency  Reports  information  to  the  Qualified 
Products  List  managers  at  the  Defense  Supply  Center  Columbus,  the  Defense 
Logistics  Agency,  and  the  Military  Departments  to  update  the  Qualified  Products 
List. 

Management  Comments.  DLA  partially  concurred  with  this  recommendation,  stating 
that  this  finding  was  not  a  specific  requirement  of  the  QPL  and  QML  Program.  DLA 
believes  that  the  PQDR  feedback  to  the  QPL  and  QML  manager  adds  value  to  the  QPL 
and  QML  Program.  They  also  believe  that  the  PQDR  feedback  will  help  the  inventory 
control  point  in  obtaining  remedial  action  by  suppliers  of  defective  parts  sold  to  DLA. 
The  DSC  Columbus  Policy  and  Plans  Division  (DSCC-BDT)  began  sending  a  list  of 
opened  and  closed  PQDRs  on  QPL  and  QML  items  with  its  associated  NSNs  grouped 
by  Federal  Stock  Classes  to  the  QA  in  November  2001.  The  list  was  distributed  to  the 
QA  Team  Chiefs  responsible  for  the  affected  QPL  and  QML  items.  The  QPL  and 
QML  engineers  will  research  the  PQDRs  to  determine  whether  the  affected  QPL  and 
QML  part  has  defects  that  would  warrant  actions  against  the  QPL  and  QML 
manufacturer  or  distributor.  If  warranted,  the  engineer  will  be  able  to  take  actions  to 
stop  a  company’s  QPL  and  QML  shipments  of  the  affected  product,  issue  problem 
reports,  and/or  remove  the  company  from  the  QPL  and  QML.  The  QPL  and  QML 
engineers  will  also  work  with  the  QAS  to  help  in  obtaining  remedial  consideration  for 
defective  parts  or  replace  defective  parts.  The  PQDR  data  is  then  input  to  the  master 
DSC  Columbus  QPL  and  QML  database  for  future  reference  and  determination  of 
whether  a  negative  trend  is  developing  with  a  particular  supplier. 

Evaluation  Response.  The  use  of  deficiency  reporting  information  provided  by  the 
user  is  an  essential  element  of  maintaining  quality  levels  with  the  QPL  and  QML 
Program.  The  PQDRs  are  the  primary  source  of  user  feedback  on  quality.  Further, 
DSC  Columbus  Integrated  Policy  Memorandum  IPM-97-0048  details  the  programs  use 
of  PQDRs.  Our  finding  on  PQDR  feedback  was  also  confirmed  in  a  DSC  Columbus 
Internal  Review  audit  report  (DSCC-DI  19-01  report  dated  November  16,  2001).  The 
reported  corrective  actions  underway  meet  the  intent  of  our  recommendation. 

3.  Provide  Certificate  of  Conformance  and  Traceability  information  to  the 
Qualified  Products  List  managers  at  the  Defense  Supply  Center  Columbus,  the 
Defense  Logistics  Agency,  and  the  Military  Departments  to  determine  authenticity 
of  a  product. 

Management  Comments.  DLA  partially  concurred  with  this  recommendation,  stating 
that  this  finding  is  not  specific  of  the  QPL  and  QML  Program  per  se.  The  COCT  is  a 
requirement  in  several  military  high-reliability  specifications  to  help  ensure  that 
authentic  QPL  and  QML  parts  are  received  when  ordered.  DLA  concurred  that  DSC 
Columbus  must  receive  COCTs  whenever  they  are  specified  in  a  requirement  document 
(i.e.,  military  specification),  and  invoked  in  the  contract.  The  QA  at  DSC  Columbus 
was  in  the  process  of  taking  corrective  action  before  the  Office  of  the  Inspector  General 
of  the  Department  of  Defense  review. 
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The  following  actions  have  already  been  implemented  at  DSC  Columbus.  A  local 
contract  clause  (E-25)  has  been  revised  to  be  sure  the  requirement  to  submit  a  COCT  is 
clear  to  the  suppliers,  and  the  buying  directorates  have  established  a  point  of  contact  to 
track  the  receipt  of  the  COCTs.  The  QAS  at  the  buying  directorates  will  review  the 
COCTs  for  adequacy  and  correctness.  In  addition,  the  E-25  clause  was  changed  to 
require  the  suppliers  to  provide  a  copy  of  the  COCT  to  the  QPL  and  QML  managers. 
This  will  allow  the  QPL  and  QML  managers  to  spot  check  to  make  sure  that  COCTs 
are  received  and  that  parts  themselves  have  been  qualified. 

Evaluation  Response.  DLA  concurred  that  DSC  Columbus  must  receive  COCTs 
whenever  they  are  specified  in  a  requirement  document,  and  invoked  in  the  contract. 

As  stated  in  the  finding,  DSC  Columbus  could  not  trace  sources  and  authenticity  of 
their  products  because  they  did  not  receive  COCTs.  User  feedback  including  COCTs 
will  facilitate  corrective  actions.  The  actions  implemented  above  meet  the  intent  of  our 
recommendation . 

4.  Perform  a  review  of  resources  required  to  accomplish  the  recertification  and 
provide  adequate  resources  to  accomplish  the  mission. 

Management  Comments.  DLA  concurred  with  this  recommendation,  stating  that  due 
to  resource  constraints  and  DoD  imposed  resource  limits,  DSC  Columbus  had  to  adjust 
their  resources.  Consequently,  the  DSC  Columbus  QA  has  to  prioritize  the  periodic 
product  qualification  audits  and  adjust  audit  frequencies.  They  proposed  to  review 
engineering  resources,  review  recruitment  actions  to  fill  critical  engineering  vacancies, 
and  increase  automation  to  improve  efficiency  and  effectiveness. 

Evaluation  Response.  In  response  to  the  final  report,  we  request  that  DLA  provide 
completion  dates  for  the  review  of  engineering  resources  required  to  accomplish  the 
certifications. 
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Appendix  A.  Evaluation  Process 


Scope  and  Methodology 


To  accomplish  the  evaluation  objective,  we  examined  DSC  Columbus  management  of  the 
QPL  and  QML  Program  including  qualification  process,  maintenance  of  specifications  and 
lists  of  contractors,  and  its  response  to  quality  related  problems  associated  with  QPL  and 
QML  items. 

We  reviewed  the  DSC  Columbus  organizational  structure,  QPL  and  QML  related 
maintenance  actions,  and  reported  deficiencies  against  qualified  products.  We  obtained 
and  reviewed  DSC  Columbus’s  4  QMLs  and  300  QPLs  containing  186,000  NSNs  and 
analyzed  data  from  3  deficiency  reporting  programs:  PQDR,  Product  Verification 
Program,  and  Government  Industry  Data  Exchange  Program.  We  evaluated  the  QPL  and 
QML  Program’s  retention  of  qualification  requirements  managed  by  DSC  Columbus. 
Results  of  the  evaluation  provided  insight  into  the  effectiveness  of  the  program  and  the 
adequacy  of  its  quality  control  efforts. 

General  Accounting  Office  High-Risk  Area.  The  General  Accounting  Office  has 
identified  several  high-risk  areas  in  DoD.  This  evaluation  provides  coverage  of  the  DoD 
Inventory  Management  high-risk  area. 

Use  of  Computer-Processed  Data.  To  achieve  the  evaluation  objective,  we  used  the 
DSC  Columbus  Internal  Review  Audit  Office  statement  of  the  number  of  PQDRs  received 
by  DSC  Columbus  that  were  associated  with  QPL  and  QML  NSNs.  The  statement  was 
based  on  their  analysis  of  both  active  and  inactive  databases.  We  did  not  establish  the 
reliability  of  the  computer  databases  and  nothing  came  to  our  attention  that  caused  us  to 
doubt  the  reliability  of  their  databases.  However,  establishing  the  reliability  of  the 
databases  will  not  materially  affect  the  results  of  our  evaluation  because  the  issue  was  not 
based  upon  the  exactness  of  the  number. 

Evaluation  Dates  and  Standards.  We  performed  this  evaluation  from  March  2001 
through  September  2001  according  to  standards  implemented  by  the  Inspector  General  of 
the  Department  of  Defense. 

Contacts  During  the  Evaluation.  We  visited  or  contacted  individuals  and  organizations 
within  DoD.  Further  details  are  available  on  request. 


Management  Control  Program  Review 


DoD  Directive  5010.38,  “Management  Control  (MC)  Program,”  August  26,  1996,  and 
DoD  Instruction  5010.40,  “Management  Control  (MC)  Program  Procedures,” 

August  28,  1996,  require  DoD  organizations  to  implement  a  comprehensive  system  of 
management  controls  that  provides  reasonable  assurance  that  programs  are  operating  as 
intended  and  to  evaluate  the  adequacy  of  the  controls. 

Scope  of  the  Review  of  the  Management  Control  Program.  We  reviewed  the 
management  control  program  related  to  the  overall  evaluation  objectives  and  determined 
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that  the  pertinent  management  controls  concerning  the  QPL  and  QML  Program  were 
inadequate  (see  the  Finding). 

Adequacy  of  Management  Controls.  We  identified  material  management  control 
weaknesses  at  DSC  Columbus  as  defined  by  DoD  Instruction  5010.40.  DSC  Columbus 
controls  over  the  QPL  and  QML  Program  were  not  adequate  to  insure  compliance  with 
regulations  and  management  controls  for  the  certification  and  retention  of  its  qualified 
manufacturers.  In  addition,  the  Sourcing  and  Qualifications  Unit  could  monitor  neither 
PQDR  nor  COCT  documentation  because  the  appropriate  mechanisms  had  not  been 
instituted  between  those  receiving  PQDRs  and  COCTs  and  those  maintaining  the  QPL  and 
QML.  If  management  implements  all  recommendations,  the  management  control 
weaknesses  will  be  corrected.  A  copy  of  the  report  will  be  provided  to  the  senior  official 
responsible  for  management  controls  within  DSC  Columbus. 

Adequacy  of  Management’s  Self-Evaluation.  DSC  Columbus  management  did  not 
identify  management  of  the  QPL  and  QML  Program  as  an  assessable  unit  and,  therefore, 
did  not  identify  or  report  the  QPL  and  QML  Program  management  control  weaknesses 
identified  by  the  evaluation. 

Prior  Coverage 

Inspector  General  of  the  Department  of  Defense  (IG  DoD) 

IG  DoD  Report  No.  D-2002-080,  “Quality  Deficiency  Reporting  Procedures  for  Naval 
Repair  Parts,”  April  5,  2002 

IG  DoD  Report  No.  D-2001-054,  “Defense  Logistics  Agency  Product  Verification 
Program,”  February  21,  2001 

IG  DoD  Report  No.  D-200 1-002,  “Defense  Logistics  Agency  Customer  Returns 
Improvement  Initiative  Program,”  October  12,  2000 

IG  DoD  Report  No.  98-063,  “Defense  Logistics  Agency  Product  Quality  Deficiency 
Program,”  February  5,  1998 
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Appendix  B.  Definitions  of  Technical  Terms 


Certification  of  Conformance  and  Traceability.  Certification  is  a  written  document 
described  in  a  specification,  signed  by  the  company  official  for  a  product  supplied, 
manufactured,  and  tested  in  full  compliance  with  the  specification  requirements.  The 
manufacturer’s  certificate  is  sent  with  each  shipment  to  a  customer  to  ensure  that  complete 
product  compliance  and  traceability  from  the  original  manufacturer  is  maintained  through 
any  intermediate  sources  such  as  a  distributor  or  dealer  to  a  customer. 

Distributor.  Anyone  authorized  by  the  manufacturer  to  distribute  the  manufacturer’s 
product.  This  includes  the  distributor  authorized  by  the  manufacturer  to  rebrand  and 
distribute  the  manufacturer’s  rebrand  product  under  the  distributor’s  own  brand. 

Facility.  One  or  more  buildings  or  plant  sites  (usually  co-located)  used  to  design, 
manufacture  and/or  test  components. 

Hybrids  Manufacturing  Line.  DSC  Columbus  considers  the  substrate  fabrication  area  to 
be  one  manufacturing  line,  the  assembly  area  to  be  another  line,  and  the  test  area  to  be  a 
third  line. 

Manufacturing  Line.  Primarily  refers  to  a  unique  set  of  manufacturing  processing  steps 
necessary  (usually  described  by  the  QPL  or  QML  manufacturer  as  a  manufacturing  flow) 
to  produce  a  given  technology  and/or  product  type. 

Microcircuits  Manufacturing  Lines.  Manufacturing  lines  are  segmented  by  the 
following  functional  areas:  design,  wafer  fabrication,  assembly,  and  test.  There  is 
typically  a  count  of  three  or  more  lines  per  facility  audit. 

Preparing  Activity.  The  DoD  activity  responsible  for  the  preparation,  coordination, 
issuance,  and  maintenance  of  standardization  documents. 

Qualification.  A  process  in  advance  of,  and  independent  of,  an  acquisition  by  which  a 
manufacturer’s  capabilities  or  a  manufacturer’s  or  distributor’s  products  are  examined, 
tested,  and  approved  to  be  in  conformance  to  the  applicable  specification  requirements, 
and  subsequent  approval  for  a  listing  of  products  on  a  QPL,  or  manufacturers  on  a  QML. 

Qualifying  Activity.  An  activity  that  is  either  the  preparing  activity  or  the  adopting 
activity  of  the  specification  or  its  designated  agent,  as  specified  in  the  specification  or  as 
directed  by  the  national  qualification  agency. 

Qualified  Manufacturers  List.  A  list  of  manufacturer  facilities  that  have  been  evaluated 
and  determined  to  be  acceptable  based  on  the  testing  and  approval  of  a  sample  specimen 
that  conforms  to  the  applicable  specification.  A  Qualified  Manufacturers  List  focuses  on 
qualifying  a  manufacturer’s  material  and  process  rather  than  products. 

Qualified  Products  List.  A  list  of  products  that  have  met  the  qualification  requirements 
stated  in  the  applicable  specification.  The  entry  for  each  listed  item  includes  the 
appropriate  product  identification,  the  test  or  qualification  reference,  and  the  name  and 
plant  address  of  the  manufacturer  and  distributor. 

Supplier.  Final  source  of  a  product  to  a  customer  that  may  be  identified  as  a  distributor, 
manufacturer,  or  other  entity. 
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Appendix  C.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
Under  Secretary  of  Defense  (Comptroller) 

Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 


Department  of  the  Army 

Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Naval  Inspector  General 

Auditor  General,  Department  of  the  Navy 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Air  Force 


Other  Defense  Organizations 

Director,  Defense  Logistics  Agency 

Commander,  Defense  Supply  Center,  Columbus 


Non-Defense  Federal  Organization 

Office  of  Management  and  Budget 
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Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Government  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Services 
House  Committee  on  Government  Reform 

House  Subcommittee  on  Government  Efficiency,  Financial  Management,  and 
Intergovernmental  Relations,  Committee  on  Government  Reform 
House  Subcommittee  on  National  Security,  Veterans  Affairs,  and  International  Relations, 
Committee  on  Government  Reform 

House  Subcommittee  on  Technology  and  Procurement  Policy,  Committee  on  Government 
Reform 
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Defense  Logistics  Agency  Comments 


IN  REPLY  J-33 
REFER  TO 


DEFENSE  LOGISTICS  AGENCY 
HEADQUARTERS 

8725  JOHN  J.  KINGMAN  ROAD,  SUITE  2533 
FT.  BELVOIR,  VIRGINIA  22060-6221 

MAR  I  4  2002 


MEMORANDUM  FOR  ASSISTANT  INSPECTOR  GENERAL  FOR  AUDITING 
DEPARTMENT  OF  DEFENSE 


SUBJECT:  Draft  DoD  IG  Audit  Report  on  the  Evaluation  of  the  Defense  Supply  Center 

Columbus’s  Qualified  Products  List  and  Qualified  Manufacturers  List  Program, 
Project  No.  D200IPT-0023.001,  dated  February  1, 2002 


The  Defense  Logistics  Agency  partially  concurs  with  all  recommendations  in  the  subject 
draft  report  on  Defense  Supply  Center  Columbus  (DSCC),  and  proposes  to  take  corrective 
measures  as  stated  in  Attachment  1 .  General  comments  to  the  draft  report’ s  conclusion 
concerning  the  effective  implementation  of  the  Department  of  Defense  (DoD)  Qualified 
Products  List  (QPL)  and  Qualified  Manufacturers  List  (QML)  Program  at  DSCC  are  also 
included  in  Attachment  1 .  Specific  comments  regarding  the  factual  accuracy  of  certain  parts 
of  the  draft  report  is  in  Attachment  2. 

The  DoD  QPL/QML  Program  is  important  to  the  fulfillment  of  DLA’s  mission.  The 
program  helps  increase  the  assurance  that  the  warfighters  receive  quality  parts.  We 
appreciate  the  opportunity  to  comment  on  the  draft  report.  Questions  may  be  referred  to 
Mr.  William  Lee,  Technical  and  Quality  Policy  Division,  (703)  767-1641;  or  Ms.  Annell 
Williams,  Internal  Review  office,  (703)  767-6274. 


Major  General,  USA 
Director 

Logistics  Operations 


Attachments 
DLA  Comments 
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Final  Report 
Reference 


DRAFT  EVALUATION  REPORT 
OF  THE  DEFENSE  SUPPLY  CENTER  COLUMBUS 
QUALIFIED  MANUFACTURERS  LIST  and  QUALIFIED  PRODUCTS  LIST 
PROGRAM  (PROJECT  NO.  D200IPT-0023.001,  dated  February  1, 2002) 

General  Comments: 

The  Defense  Logistics  Agency  (DLA)  partially  concurs  with  the  findings  in  the  subject  report. 
However,  we  do  not  concur  with  the  statement  in  the  executive  summary  of  the  report: 

“DSCC  did  not  have  an  effective  manufacturing  and  product  qualification  program:  . .  .’Three 
reasons  were  stated  for  this  concluding  statement:  (1)  DSCC  did  not  fully  enforce  the 
certification  and  retention  of  qualification  policy,  (2)  DSCC  did  not  receive  1,739  Product 
Quality  Deficiency  Reports  (PQDRs)  for  review,  and  (3)  DSCC  could  not  trace  sources  and 
authenticity  of  their  products  because  they  did  not  receive  6,479  Certificates  of  Conformances 
Traceability  (COCTs).  In  order  to  discuss  and  clarify  these  issues,  the  following  policy 
requirements  need  to  be  defined: 

Retention  of  Qualification:  A  set  of  detailed  technical  requirements  in  the  specification  that 
have  been  developed  and  coordinated  with  the  military  services  for  maintaining 
manufacturers  on  a  Qualified  Manufacturers  List  and  Qualified  Products  List  (QML/QPL)  for 
a  particular  technology  area.  The  requirements  are  designed  to  assure  continuous  product 
performance,  quality  and  reliability.  The  retention  of  qualification  requirements  may  include 
such  things  as  product  failure  reports,  annual  production  and  testing  reports,  critical  design 
and  process/material  changes  notices,  etc.  The  retention  requirements  can  be  used  to 
accomplish  the  certification  of  qualification  status. 

Validation:  The  process  by  which  the  Preparing  Activity  reviews  the  specifications  having 
requirements  for  qualification  every  two  years  to  determine  the  need  to  continue  qualification 
requirement. 

Certification  of  Qualification  Status:  This  is  accomplished  during  the  above  two-year  to 
validation  review  to  determine  if  a  manufacturer  is  still  a  valid  source  of  supply  in  order  be 
retained  on  the  QML/QPL.  The  certification  is  used  when  the  applicable  specification  does 
not  contain  retention  of  qualification. 

Facility  Audits:  The  requirement  for  a  facility  or  plant  site  audit  is  specified  in  the 
applicable  military  specification  and  in  paragraph  AP2.5.5  of  DoD  4120.24-M,  Defense 
Standardization  Program  Policies  and  Procedures.  Typically,  the  audit  requirement  is 
necessary  for  complex  and  critical  product  types  used  in  weapon  platforms.  The  audit 
requirement  normally  includes  a  review  of  the  suppliers’  inspection  systems,  quality  and 
reliability  assurance  programs,  processes  and  materials,  test  equipment,  test  procedures, 
training  programs,  product  traceability,  etc.  The  audits  help  mitigate  the  risk  of 
nonconforming  products,  and  assure  compliance  to  the  military  specification  requirements  as 
well  as  to  the  manufacturers’  internal  processing  requirements  for  these  complex  and  critical 
components  destined  for  military  applications.  Audits  basically  fall  under  three  broad 
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categories,  (1)  an  initial  (first-time)  audit  that  is  needed  to  determine  whether  a  new 
competitive  source  of  supply  can  be  qualified,  (2)  a  problem/fraud  audit  that  is  unplanned  but 
necessary  to  resolve  special  technical  or  non-compliance  issues  that  arise,  and  (3)  a  periodic 
re-audit  that  is  specified  in  the  applicable  military  specification  in  order  to  assure  continuous 
compliance  with  the  military  performance,  quality  and  reliability  requirements. 

DSCC  is  assigned  responsibility  to  administer  232  QPLs  and  four  QMLs.  The  validation,  and 
certifications  of  qualification  status  are  implemented  at  DSCC  through  the  detailed  retention  of 
qualification  requirements  that  are  delineated  in  the  applicable  specifications.  When  the 
certifications/validations  are  performed,  their  completion  dates  are  shown  on  the  QMLs/QPLs. 
The  QMLs/QPLs  are  then  accordingly  revised  and  posted  on  the  Department  of  Defense  (DoD) 
Acquisition  Streamline  and  Standardization  Information  System  (ASSIST)  database.  To  this  end, 
the  ASSIST  will  show  that  DSCC  has  properly  maintained  the  two-year  certification/validation 
reviews  required  by  policy. 

In  fact,  DSCC  engineers/technicians  perform  a  host  of  required  retention  of  qualification 
actions  necessary  to  properly  and  effectively  maintain  the  Product  Qualification  Program. 

These  actions  are  delineated  in  the  applicable  military  specifications,  SD-6  (Provisions 
Governing  Qualification),  and  DoD  policy  (DoD  4120.24-M).  Some  of  the  following  actions 
implemented  at  DSCC  are: 

1 .  Initial  Plant  Product  Qualification  Audits 

2.  Engineering  Evaluation  of  Authorizations  to  Test 

3.  Engineering  Evaluation  of  Product  Qualification  Data 

4.  Product  Design/Process  Change  Control  Evaluations 

5 .  Review  Annual  Retention  (Production)  of  Qualification  Reports 

6.  Periodic  Certification  of  the  Qualification  and  Manufacturers 

7.  Periodic  Product  Qualifications  Plant  Re-audits 

To  quantify  the  technical  actions  during  the  last  Calendar  Year  (CY),  DSCC  reviewed  329 
Product  Qualification  Reports  and  5,330  Product  Design/Process  Change  Reports  in  order  to 
maintain  proper  configuration  control  on  the  product,  and  evaluated  632  Annual  Retention 
Reports  last  year  in  order  to  maintain  manufacturers  and  their  product  listings  on  the 
applicable  QML/QPL. 

We  do  understand  the  importance  of  conducting  facility  audits/  The  audits  as  delineated  in  the 
specification  substantiate  the  retention  of  qualification  requirements,  and  is  independent  of 
the  “certification”  of  manufacturers  to  remain  on  the  QML/QPL.  Currently,  DSCC  schedules 
and  conducts  their  facility  (plant)  audits  within  resource  allocations  and  established  priorities 
as  follows: 
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Priority  1  -  Problem/Fraud  Audits 

Priority  2  -  Initial  Audits  for  High-Reliability  specifications 
Priority  3  -  Other  initial  audits 

Priority  4  -  Periodic  re-audits  for  High-Reliability  specifications 
Priority  5  -  Other  periodic  re-audits 

Priority  6  -  Conventional  audits  (scheduled  when  auditors  are  in  the  area) 

In  addition,  they  have  implemented  many  new  initiatives  to  further  augment  their  audit 
program  and  to  minimize  any  risk  to  the  military  services  or  industrial  market  place  when 
facility  audit  cycles  are  adjusted.  DoD  policy  does  not  allow  the  Qualifying  Activity  (QA) 
(DSCC  in  this  case)  to  remove  a  manufacturer  and  their  products  from  a  QML/QPL  solely 
because  the  QA  did  not  perform  a  facility  audit. 

We  also  would  like  to  point  out  that  two  of  the  audit  findings,  i.e.  regarding  PQDRs  and 
COCTs,  are  not  specific  requirements  for  a  product  qualification  program  per  se  but  rather  are 
provisions  whereby  the  QML/QPL  manager,  if  provided  with  such  information,  could  help 
DSCC  or  other  buying  activities  in  obtaining  authentic  QML/QPL  products  and  resolving 
quality  issues. 

We  believe  the  effectiveness  of  the  QML/QPL  Program  should  be  based  on  how  well  the 
QML/QPL  requirements  facilitate  the  procurement  process  by  qualifying  parts  and/or 
manufacturers  in  advance  of  actual  procurements  or  contracts,  thereby  avoiding  delays  during 
the  procurement  processes  while  reducing  the  need  for  other  duplicative  contract  provisions. 

Specific  comments  on  the  recommendations: 

Recommendation  No.  1:  Fully  implement  the  procedures  specified  in  DoD  Manual  4120.24- 
M,  “Defense  Standardization  Program  (DSP)  Policies  and  Procedures,”  March  2000,  for 
Qualified  Products  List  and  Qualified  Manufacturers  List  Programs,  to  either  recertify  the 
manufacturer  within  the  required  cycle  or  remove  the  manufacturer  from  the  Qualified 
Products  List  and  Qualified  Manufacturers  List  Program. 

DLA  COMMENT:  Partially  Concur. 

DoD  policy  delineates  conditions  for  removal  of  QML/QPL  sources,  none  of  which  include  a 
provision  for  a  QA  to  remove  manufacturers  from  a  QML/QPL  solely  because  the  QA  did  not 
perform  a  facility  (plant)  audit,  nor  “recertify”  the  manufacturer  within  the  required  cycle. 

As  indicated  in  our  general  comments,  DSCC  performs  all  the  elements  necessary  to  “recertify” 
that  the  manufacturers  can  be  retained  on  the  QML/QPL.  DSCC  performs  the  certification  of  a 
manufacturer  as  part  of  the  retention  of  qualification  requirements  specified  in  our  military 
specifications.  DoD  4120.24-M  permits  this.  When  the  QA  completes  the  certification  review  of 
all  of  the  manufacturers  on  the  QML/QPL,  the  QML/QPL  is  then  revised  and  posted  on  the 
ASSIST.  The  ASSIST  database  will  show  that  DSCC  has  properly  maintained  the  two-year 
certification/validation  reviews  required  by  policy. 
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The  requirement  to  perform  facility  audits  in  conjunction  with  the  retention  of  qualification  is 
only  one  element  within  DSCC’s  overall  QML/QPL  Program  that  assures  compliance  to  the 
military  specification  requirements.  Although  audits  are  specified  in  the  applicable  military 
specifications,  they  are  not  a  prerequisite  for  the  certification  of  manufacturers  to  remain  on 
the  QML/QPL. 

In  terms  of  performing  the  specified  audits  of  the  manufacturers,  we  acknowledge  that 
qualification  audits  are  important  tools  to  mitigate  quality  problems  and  assure  continued 
product  performance,  quality  and  reliability.  DSCC  plans  their  product  qualification  audits 
within  resource  allocations  based  upon  the  prioritization  criteria,  stated  earlier,  and  manage 
risk  through  adjustments  of  the  audit  cycles.  However,  DSCC,  like  all  agencies,  has  resource 
constraints  that  must  be  managed.  DSCC  had  to  adjust  their  resources  accordingly  with  their 
needs  as  an  Inventory  Control  Point  (ICP). 

Disposition: 

(X)  Action  is  ongoing.  Engineering  resources  will  be  continually  reviewed.  See 
Recommendation  No.  4.  ECD:  90  days. 

(  )  Action  is  considered  complete. 


Recommendation  No.  2:  Provide  Product  Quality  Deficiency  Reports  information  to  the 
Qualified  Products  List  managers  at  the  Defense  Supply  Center,  Columbus,  the  Defense 
Logistics  Agency,  and  Military  Departments  to  update  the  Qualified  Products  List. 

DLA  Comments:  Partially  Concur. 

This  finding  is  not  a  specific  requirement  of  the  QML/QPL  Program.  However,  we  do 
believe  that  the  PQDR  feedback  to  the  QML/QPL  manager  adds  value  to  the  QML/QPL 
Programs.  This  information  can  provide  additional  data  to  be  used  to  adjust  the  audit 
intervals.  More  importantly,  the  PQDR  feedback  will  help  the  ICP  increase  the  in  leverage  in 
obtaining  remedial  action  for  defective  parts  sold  to  us  by  suppliers.  As  indicated  in  the  draft 
audit  report,  DSCC  has  policy  memorandum  (IPM  97-0048,  dated  May  13, 1998),  which 
requires  the  Quality  Assurance  Specialist  (QAS)  in  the  buying  directorates  to  provide  the 
QML/QPL  managers  all  PQDRs  involving  QML/QPL  product.  The  DoDIG  draft  report 
finding  on  PQDR  feedback  was  confirmed  in  a  DSCC  Internal  Review  audit  review  report 
(DSCC-DI  19-01  report  dated,  November  16,  2001).  Corrective  actions  are  underway  to 
strengthen  this  area.  The  details  of  the  corrective  actions  are: 

In  November  2001,  the  DSCC  Policy  and  Plans  Division  (DSCC-BDT)  began  sending  a  list 
of  opened/closed  PQDRs  on  QML/QPL  items  with  its  associated  National  Stock  Numbers 
(NSNs)  grouped  by  Federal  Stock  Classes  (FSCs)  to  the  QA.  The  list  is  then  distributed  to 
the  QA  Team  Chiefs  responsible  for  the  affected  QML/QPL  items.  The  QML/QPL  engineers 
then  research  the  PQDR  to  determine  whether  the  affected  QML/QPL  part  has  defects  that 
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would  warrant  actions  against  the  QML/QPL  manufacturer  or  distributor.  If  warranted,  the 
engineer  will  take  actions  to  stop  a  company's  QML/QPL  shipments  of  the  affected  product, 
issue  problem  reports  (GIDEP  Alerts),  and/or  remove  the  company  from  the  QML/QPL.  The 
QML/QPL  engineers  will  also  work  with  the  QAS  to  help  in  obtaining  remedial  consideration 
for  defective  parts  (e.g.  replace  defected  parts).  The  PQDR  data  is  then  input  to  the  master 
DSCC  QML/QPL  database  for  future  reference  and  determination  of  whether  a  negative  trend 
is  developing  with  a  particular  supplier. 

Disposition: 

(X)  Action  is  ongoing.  DSCC  Internal  Review  (DSCC-DI)  will  conduct  periodic  reviews  to 
assure  compliance  to  policy  memorandum  IPM-97-0048.  ECD:  By  the  end  of  CY  2002. 

(  )  Action  is  considered  complete. 

Recommendation  No.  3:  Provide  Certificate  of  Conformance  and  Traceability  information 
to  the  Qualified  Products  List  managers  at  the  Defense  Supply  Center  Columbus,  the  Defense 
Logistics  Agency,  and  Military  Departments  to  determine  authenticity  of  a  product. 

DLA  Comments:  Partially  Concur. 

This  finding  is  not  a  specific  requirement  of  the  QML/QPL  Program  per  se.  Rather,  the 
COCT  is  a  requirement  in  several  military  high  reliability  specifications  to  help  ensure  that 
authentic  QML/QPL  parts  are  received  when  ordered.  Verification  that  COCTs  are  received 
is  a  contractual  matter,  which  resides  in  the  DSCC  buying  directorates.  The  QA  at  DSCC 
helps  ascertain  the  authenticity  of  the  product  source,  and  its  products.  The  QA  expertise  has 
helped  DSCC  recover  roughly  $  1 90  million  from  fraud  investigations  during  the  past  1 2 
years. 

We  do  concur  that  DSCC  must  receive  COCTs  whenever  they  are  specified  in  a  requirement 
document  (i.e.,  military  specification),  and  invoked  in  the  contract.  The  QA  at  DSCC  was  in 
the  process  of  taking  corrective  action  before  the  DoDIG  review.  Soon  after  the  DoDIG 
review,  DSCC  Internal  Review  office  investigated  the  finding  and  ascertained  the  problem  to 
only  MIL-PRF- 19500  semiconductor  products.  As  the  result  the  DSCC  Internal  Review  audit 
review  (DSCC-DI  report  19-01,  dated  November  16, 2001),  the  following  actions  have 
already  been  implemented  at  DSCC  to  assure  the  buying  directorates  receive  the  COCTs  as 
required: 

1.  A  local  contract  clause  (E-25)  has  been  revised  to  be  sure  the  requirement  to  submit  a 
COCTs  is  clear  to  the  suppliers. 

2.  The  buying  directorates  have  established  a  point  of  contact  to  track  the  receipt  of  the 
COCTs.  The  QAS  at  the  buying  directorates  will  review  the  COCTs  for  adequacy  and 
correctness. 
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3.  The  QML/QPL  managers  have  sent  letters  to  all  DSCC  suppliers  of  products  requiring 
COCTs  to  remind  them  of  the  necessity  of  adhering  to  the  COCT  requirements. 

4.  The  QML/QPL  managers  have  sent  letters  to  the  Quality  Assurance  Representatives 
(QARs)  at  the  Defense  Contract  Management  Agency  to  remind  them  to  check  for  COCTs  on 
source-inspected  parts.  The  QARs  have  been  encouraged  to  report  any  problems  to  DSCC. 

5.  The  E-25  clause  was  also  changed  to  require  the  suppliers  to  provide  a  copy  of  the  COCT 
to  the  QML/QPL  managers.  This  will  allow  the  QML/QPL  managers  to  spot  check  to  make 
sure  we  are  getting  COCTs  and  that  the  parts  themselves  have  been  qualified. 

Disposition: 

(X)  Action  is  ongoing.  DSCC  Internal  Review  (DSCC-DI)  will  conduct  periodic  reviews  to 
assure  that  the  COCTs  are  being  received  at  DSCC.  ECD:  By  the  end  of  CY  2002. 

(  )  Action  is  considered  complete. 

Recommendation  No.  4:  Perform  a  review  of  resources  required  to  accomplish  the 
recertification  and  provide  adequate  resources  to  accomplish  the  mission. 

DLA  Comments:  Concur. 

The  qualification  process  requires  a  resource  commitment  to  effectively  maintain  performance, 
quality  and  reliability  of  the  QML/QPL  products.  DSCC  is  currently  reviewing  recruitment 
actions  to  fill  critical  engineering  vacancies.  The  DoD  QML/QPL  Program  is  important  to 
DLA’s  mission  and  assures  that  the  war  fighter  receives  the  right  item,  the  right  time,  the  right 
place,  and  the  right  price  every  time.  Every  year,  there  are  new  sources  that  have  to  be  audited 
to  determine  whether  they  can  be  qualified.  Furthermore,  the  QA  must  address  fraud 
problems,  problem  sources,  and  sources  that  change  their  manufacturing  process,  or  move  their 
manufacturing  facility,  which  all  requires  audits.  To  accomplish  this  workload,  dedicated 
engineers  positions  have  been  assigned.  To  this  end,  the  program  has  been  taken  very 
seriously  at  DSCC. 

Due  to  the  resource  constraints  and  the  DoD  imposed  resource  limits,  which  face  each 
Agency  and  Military  Department  every  year,  DSCC  had  to  adjust  their  resources  accordingly 
with  the  ICP’s  needs.  Consequently,  the  DSCC  QA  has  to  prioritize  the  periodic  product 
qualification  audits,  and  adjust  audit  frequencies  accordingly.  DSCC  has  implemented 
changes  to  several  specifications  to  minimize  risk  for  the  military  services  and  the  war  fighter 
from  receiving  poor  quality  or  non-conforming  parts. 
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Also,  improvements  in  efficiency  and  effectiveness  are  possible  through  increased 
automation.  In  December  2000,  the  DoD  Qualification  Working  Group  (QWG)  identified 
several  areas  where  automation  would  improve  processes  and  use  resources  more  effectively. 
The  Defense  Standardization  Program  Office  (DSPO)  accepted  the  QWG  recommendation  to 
redesign  the  qualification  lists  and  related  databases  and  automate  the  QML/QPL 
administration  process  to  the  degree  possible.  The  redesign  will  add  new  automated 
capabilities  that  will  simplify  and  streamline  the  qualification  system.  DSPO  will  implement 
this  change  over  a  thirty-month  transition  period.  In  fact,  DSPO  is  working  with  DSCC  to 
model  the  redesign  of  their  current  automated  systems,  the  ASSIST,  with  DSCC’s  current 
QML/QPL  database  functionalities.  DSCC  is  the  only  QA  within  DoD  to  have  a 
sophisticated  database  to  help  manage  the  QML/QPL  Program. 

Disposition: 

(X)  Action  is  ongoing.  Recruitment  actions  to  fill  critical  engineering  vacancies  will  be 
reviewed.  ECD:  90  days. 

(  )  Action  is  considered  complete. 
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DRAFT  EVALUATION  REPORT 

EVALUATION  OF  THE  DEFENSE  SUPPLY  CENTER  COLUMBUS 
QUALFIED  MANUFACTURERS  LIST  and  QUALIFIED  PRODUCTS  LIST  PROGRAM 
(PROJECT  NO.  D200IPT-0023.001,  dated  February  1, 2002) 

Corrections  for  Accuracy 

1 .  Page  i,  Executive  Summary,  Results.  First  bullet  -  change  “Recertification”  to 
"Certification”.  REASON:  “Certification”  is  the  terminology  used  in  DoD  4120.24-M, 
“Defense  Standardization  Program  (DSP)  Policies  and  Procedures,”  March  2000. 

2.  Page  ii.  Executive  Summary,  Summary  of  Recommendations.  Last  sentence:  Change 
“recertification”  to  “certification”.  REASON:  “certification”  is  the  terminology  used  in 
DoD  4120.24-M. 

3.  Page  2,  Background,  QPL  Review  and  Recertification.  First  sentence:  Change 
“Recertification”  to  “Certification”.  REASON:  “Certification”  is  the  terminology  used  in 
DoD  4120.24-M.  Third  sentence:  Change  “preparing  activity”  to  “Qualifying  Activity” 
and  “recertifying”  to  “certify”.  REASON:  The  Qualifying  Activity  is  responsible  for 
reviewing  and  certifying  each  QPL/QML  manufacturers  on  a  periodic  basis.  Fifth 
sentence:  Change  “recertification”  to  “certification”. 

4.  Page  3,  Background,  QML  Review  and  Recertification.  First  sentence:  Change 
“Recertification”  to  “Certification”.  REASON:  “Certification”  is  the  terminology  used  in 
DoD  4120.24-M.  Delete  second  sentence,  and  substitute:  “The  Preparing  Activity  must 
review  specifications  that  require  qualification  every  2  years  to  determine  the  need  to 
continue  the  qualification  requirement.”  REASON:  The  sentence  in  the  report  does  not 
accurately  represent  the  Preparing  Activity’s  role  as  stated  in  DoD  4120.24-M.  Third 
sentence:  Change  “preparing  activity”  to  “Qualifying  Activity”  and  “recertifying”  to 
“certify”.  REASON:  The  Qualifying  Activity  is  responsible  for  reviewing  and  certifying 
each  QPL/QML  manufacturers  on  a  periodic  basis. 

5.  Page  3,  Background,  QPL  and  QML  Program  at  Defense  Supply  Center  Columbus. 

Second  paragraph,  last  sentence:  Change  “Defense  Logistics  Agency  (DLA)  Acquisition 
Streamline  and  Standardization  Information  System  (ASSIST)”  to  ‘Department  of  Defense 
(DoD)  Acquisition  Streamlining  and  Standardization  Information  System  (ASSIST)”. 
REASON:  (a)  The  Defense  Standardization  Program  Office  under  ADUSD  for  Logistics 
Plans  and  Programs  manages  the  ASSIST  database,  and  (b)  Correct  the  spelling  of  the 
system  name. 

6.  Page  4,  Finding,  Retention  of  Qualification  Policy.  First  and  second  sentences:  Change 
“Recertification”  to  “Certification”.  REASON:  “Certification”  is  the  terminology  used  in 
DoD  4120.24-M.  Second  sentence:  Delete  after  “status”:  “and  a  review  of  all 
specifications”.  REASON:  The  sentence  in  the  report  does  not  accurately  represent  the 
Qualifying  Activity’s  role  as  stated  in  DoD  4120.24. 
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7.  Page  5,  Recertification  Process  at  DSC  Columbus.  Change  the  paragraph  title  to  read: 
“Certification  Process  at  DSC  Columbus”.  REASON:  “Certification”  is  the  terminology 
used  in  DoD  4120.24-M.  Fifth  sentence:  Change  “recertified,”  to  “certified,”  REASON: 
“Certified”  is  the  terminology  used  in  DoD  4120.24. 

8.  Page  6,  Results  ofRecertification  Process  at  DSC  Columbus:  Change  the  paragraph  title 
to  read:  “Results  of  Retention  of  Qualification  Process  at  DSC  Columbus”.  REASON : 
The  title  accurately  reflects  the  paragraph  discussion.  Fifth  sentence:  Change  “recertified” 
to  “certified”;  REASON :  Correction  terminology  used  in  DoD  4120.24-M. 

9.  Page  6,  Results  of  Recertification  Assessment.  Chaiige  the  paragraph  title  to  read: 

“Results  of  Retention  of  Qualification  Assessment”.  REASON:  The  title  accurately 
reflects  the  paragraph  discussion.  First  paragraph,  third  sentence:  Change 
“Recertification"  to  “Certification”.  Second  paragraph,  fifth  and  sixth  sentences:  Change 
“recertification  to  “certification”.  REASONS:  Correction  terminology  used  in  DoD 
4120.24-M. 

10.  Page  7,  DSC  Columbus  Sourcing  and  Qualification  Unit  Personnel  and 
Workload.  Last  line:  Change  “recertification  to  “certification”.  REASON: 

Correction  terminology  used  in  DoD  4120.24-M. 
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